AI621 A3 : Image Blending

20215584 Wonjoon Chang

Youtube: https://youtu.be/gUB_7NJi8VY

(2) Poisson Blending

Choose the source image and the target image (background).

source: (1) penguin/penguin-chick (2) duck (3) ufo (4) ghost

target: (1) hiking (2) Hallstatt (3) Shanghai (4) Kyoto

im_background = imresize(im2double(imread('./data/hiking.jpg')), 0.5, 'bilinear');
im_object = imresize(im2double(imread('./data/penguin.jpg')), 0.5, 'bilinear');
% im_object = imresize(im2double(imread('./data/penguin-chick.jpeg')), 0.5, 'bilinear');
 

Masking for Aligned and Resized Image

Get a source region mask from the user.

I revised getMask function to obtain the polygon information.

[objmask, sx, sy] = getMask(im_object);

Draw polygon around source object in clockwise order, q to stop

Align im_s and mask_s with im_background.

I also revised alignSource function to enable to resize the source image. The second point determine the top location for rescaling. In this process, the polygon information is used, which we obtained in the previous step.

[im_s, mask_s] = alignSource(im_object, objmask, sx, sy, im_background);

choose target bottom-center location
choose target top location for scaling

Solving the least square problem

Note that there are additional constraints for the seamless boundary unlike the toy problem. These constraints are reflected in poissonBlend_black.m. Since we now deal with RGB colored images, we should solve the least square problem for each color and combine the results.

v_1 = poissonBlend_black(mask_s, im_s(:,:,1), im_background(:,:,1));
v_2 = poissonBlend_black(mask_s, im_s(:,:,2), im_background(:,:,2));
v_3 = poissonBlend_black(mask_s, im_s(:,:,3), im_background(:,:,3));

imh = size(im_s, 1);
imw = size(im_s, 2);

v_list = [v_1, v_2, v_3];
im_blend = zeros(size(im_background));
 
for i = 1:3
    tmp_blend = zeros(imh,imw,1);
    tmp_blend(:,:,:) = im_background(:,:,i);
    tmp_blend(mask_s) = v_list(:,i);
    im_blend(:,:,i) = tmp_blend(:,:,:);
end 
 

figure(3), hold off, imshow(im_blend)

Blending with Mixed Gradients

We can also use the gradient in source or target with the larger magnitude as the guide. The objective function follows as:

The new image will can reflect the gradients of the background image, which has some advantages if you want to consider translucent things.

v_1 = poissonBlend_mixed_black(mask_s, im_s(:,:,1), im_background(:,:,1));
v_2 = poissonBlend_mixed_black(mask_s, im_s(:,:,2), im_background(:,:,2));
v_3 = poissonBlend_mixed_black(mask_s, im_s(:,:,3), im_background(:,:,3));

v_list = [v_1, v_2, v_3];
im_mixed_blend = zeros(size(im_background));
 
for i = 1:3
    tmp_blend = zeros(imh,imw,1);
    tmp_blend(:,:,:) = im_background(:,:,i);
    tmp_blend(mask_s) = v_list(:,i);
    im_mixed_blend(:,:,i) = tmp_blend(:,:,:);
end 

figure(3), hold off, imshow(im_mixed_blend)

Results

penguin/penguin-chick + hiking

There are some differences between two approaches.

When using the source gradients, the remaining part in the masked area, which means the part does not contain the object, may result in unnatural seams. In these cases, it doesn't matter, because the backgrounds of the source image and the target image have similar color ranges.

When using mixed gradients, this potential problem will be alleviated. But some background components appear. It is because there are larger gradients in the target background image (snow~mountain).

duck + Hallstatt (stream)

Using the source gradients blends the images more naturally. However, mixed gradients induce a bad texture of the duck. It is because the steam has very complex texture so that it has large gradients.

ufo + Shanghai (sky)

In both cases, the ufo is attached successfully, because the sky has very small gradients. However, the color of the ufo highly depends on the color of the sky, Its color becomes blue, while the original color is close to black.

ghost + Kyoto (woods)

The final image is the ghost in the dark woods. I chose to add the ghost in order to show the effect of the mixed gradients setting. In this case, using mixed gradients makes the ghost translucent since the gradients of the background appear. It feels more scary.